Method and electronic device for switching between first lens and second lens

ABSTRACT

A method for switching between a first lens and a second lens in an electronic device includes displaying, by the electronic device, a first frame showing a field of view (FOV) of the first lens; detecting, by the electronic device, an event that causes the electronic device to transition from displaying the first frame to displaying a second frame showing a FOV of the second lens; generating, by the electronic device and based on the detecting the event, at least one intermediate frame for transitioning from the first frame to the second frame; and switching, by the electronic device and based on the detecting the event, from the first lens to the second lens and displaying the second frame, wherein the at least one intermediate frame is displayed after the displaying the first frame and before the displaying the second frame while the switching is performed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 U.S.C. § 119to an Indian Provisional Patent Application No. 201841038833 filed onOct. 12, 2018 and an Indian Complete Patent Application No.201841038833, filed on Oct. 7, 2019, the disclosures of which areincorporated by reference herein in their entirety.

BACKGROUND 1. Field

The disclosure relates to an image processing system, and morespecifically is related to a method and electronic device for switchingbetween a first lens and a second lens.

2. Description of Related Art

In general, flagship features like dual and triple cameras are beingintroduced in an electronic device (e.g., smart phone or the like). Butthere are various constraints in the implementation of the flagshipfeatures. One major constraint is that, all camera systems cannot besimultaneously kept turned on, which causes a camera switching delay.The transition between the cameras is not seamless which results inreducing the user experience.

In the existing methods, switching from one camera to another camera maybe performed when frames from both cameras are available duringtransition. A multi frame fusion module combines information from theframes and sends a resulting frame to a display for preview. This alsoresults in reducing the user experience.

Thus, it is desired to address the above mentioned disadvantages orother shortcomings or provide a useful alternative.

SUMMARY

In accordance with an aspect of the disclosure, a method for switchingbetween a first lens and a second lens in an electronic device includesdisplaying, by the electronic device, a first frame showing a field ofview (FOV) of the first lens; detecting, by the electronic device, anevent that causes the electronic device to transition from displayingthe first frame to displaying a second frame showing a FOV of the secondlens; generating, by the electronic device and based on the detectingthe event, at least one intermediate frame for transitioning from thefirst frame to the second frame; and switching, by the electronic deviceand based on the detecting the event, from the first lens to the secondlens and displaying the second frame, wherein the at least oneintermediate frame is displayed after the displaying the first frame andbefore the displaying the second frame while the switching is performed.

The generating the at least one intermediate frame may includedetermining a lens switching delay from a first time at which the firstframe is displayed to a second time at which the second frame isdisplayed; identifying at least one transition parameter of the firstlens and the second lens to generate the at least one intermediateframe; obtaining at least one from among a spatial alignment data, aphotometric alignment data and a color alignment data of the first lensand the second lens; and generating the at least one intermediate framebased on the at least one transition parameter, the lens switchingdelay, and at least one from among the spatial alignment data, thephotometric alignment data and the color alignment data.

The at least one intermediate frame may be at least one from amongspatially aligned with respect to the first frame and the second frame,photometrically aligned with respect to the first frame and the secondframe and color aligned with respect to the first frame and the secondframe.

The method may further include determining the at least one intermediateframe to be generated, wherein determining the at least one intermediateframe to be generated includes determining the spatial alignment datausing the at least one transition parameter; and determining the atleast one intermediate frame to be generated based on the determinedspatial alignment data and the at least one transition parameter.

The spatial alignment data is obtained by capturing a first single frameassociated with the first lens and a second single frame of a same sceneassociated with the second lens when the electronic device is in an idlemode; resizing the first single frame and the second single frame into apreview resolution size; computing feature points in the first singleframe and the second single frame; computing a transformation matrixusing a Homography relationship between the first single frame and thesecond single frame, wherein the transformation matrix includes a firstscaling of the first single frame, a second scaling of the second singleframe, a first rotation of the first single frame, a second rotation ofthe second single frame, a first translation of the first single frame,and a second translation of the second single frame; and obtaining thespatial alignment data using the transformation matrix.

The photometric alignment data may be obtained by computing atransformation matrix for the generated at least one intermediate frame;computing a correction factor based on the transformation matrix; andobtaining the photometric alignment data based on the correction factor.

The color alignment data may be obtained by computing a transformationmatrix for the generated at least one intermediate frame; computing acorrection factor for the color alignment data based on thetransformation matrix; and obtaining the color alignment data based onthe correction factor.

The at least one transition parameter may include an F-value of thefirst lens, the FOV of the first lens, a color profile of the firstlens, a saturation profile of the first lens, an F-value of the secondlens, the FOV of the second lens, a color profile of the second lens, asaturation profile of the second lens, a scale factor of the first lens,a scale factor of the second lens, a scale factor between the first lensand the second lens, a single scale factor of a combination of the firstlens and the second lens, a pivot between the first lens and the secondlens, and a single pivot value of a combination of the first lens andthe second lens.

In accordance with an aspect of the disclosure, an electronic device forswitching between a first lens and a second lens includes a memory; aprocessor coupled with the memory, the processor being configured todisplay a first frame showing a field of view (FOV) of the first lens;detect an event that causes the electronic device to transition fromdisplaying the first frame to displaying a second frame showing a FOV ofthe second lens; generate, based on detecting the event, at least oneintermediate frame for transitioning from the first frame to the secondframe; and switch, based on detecting the event, from the first lens tothe second lens and display the second frame, wherein the at least oneintermediate frame is displayed after the first frame is displayed andbefore the second frame is displayed while the switching is performed.

The processor may be further configured to determine a lens switchingdelay from a first time at which the first frame is displayed to asecond time at which the second frame is displayed; identify at leastone transition parameter of the first lens and the second lens togenerate the at least one intermediate frame; obtain at least one fromamong a spatial alignment data, a photometric alignment data and a coloralignment data of the first lens and the second lens; and generate theat least one intermediate frame based on the at least one transitionparameter, the lens switching delay, and at least one from among thespatial alignment data, the photometric alignment data and the coloralignment data.

The at least one intermediate frame may be at least one from amongspatially aligned with respect to the first frame and the second frame,photometrically aligned with respect to the first frame and the secondframe and color aligned with respect to the first frame and the secondframe.

The processor may be further configured to determine the spatialalignment data using the at least one transition parameter; anddetermine the at least one intermediate frame to be generated based onthe determined spatial alignment data and the at least one transitionparameter.

The processor may be further configured to capture a first single frameassociated with the first lens and a second single frame of a same sceneassociated with the second lens when the electronic device is in an idlemode; resize the first single frame and the second single frame into apreview resolution size; compute feature points in the first singleframe and the second single frame; compute a transformation matrix usinga Homography relationship between the first single frame and the secondsingle frame, wherein the transformation matrix includes a first scalingof the first single frame, a second scaling of the second single frame,a first rotation of the first single frame, a second rotation of thesecond single frame, a first translation of the of the first singleframe, and a second translation of the second single frame; and obtainthe spatial alignment data using the transformation matrix.

The photometric alignment data may be obtained by computing atransformation matrix for the generated at least one intermediate frame;computing a correction factor based on the transformation matrix; andobtaining the photometric alignment data based on the correction factor.

The color alignment data may be obtained by computing a transformationmatrix for the generated at least one intermediate frame; computing acorrection factor for the color alignment data based on thetransformation matrix; and obtaining the color alignment data based onthe correction factor.

The at least one transition parameter may include an F-value of thefirst lens, the FOV of the first lens, a color profile of the firstlens, a saturation profile of the first lens, an F-value of the secondlens, the FOV of the second lens, a color profile of the second lens, asaturation profile of the second lens, a scale factor of the first lens,a scale factor of the second lens, a scale factor between the first lensand the second lens, a scale factor of a combination of the first lensand the second lens, a pivot between the first lens and the second lens,and a single pivot value of a combination of the first lens and thesecond lens.

In accordance with an aspect of the disclosure, a device includes amemory configured to store a first attribute of a first camera and asecond attribute of a second camera; and a processor configured togenerate at least one intermediate image frame based on the firstattribute and the second attribute; output the at least one intermediateimage frame after outputting a first image frame captured by the firstcamera and before outputting a second image frame captured by the secondcamera.

The memory may be further configured to store at least one transitionparameter based on the first attribute and the second attribute, and theprocessor may be further configured to generate the at least oneintermediate image frame based on the at least one transition parameter.

The first image frame captured by the first camera and the second imageframe captured by the second camera may be of a same scene, and theprocessor may be configured to generate a transformation matrix based ona Homography relationship between the first image frame and the secondimage frame, the Homography relationship being determined based on theat least one transition parameter, and generate the at least oneintermediate image frame using the transformation matrix.

The processor may be further configured to determine a coefficient foreach intermediate image frame from among the at least one intermediateimage frame and generate each intermediate image frame based on thetransformation matrix and on the respective determined coefficient.

The memory may be further configured to store a switching delay betweenthe first camera and the second camera and a frame rate, and theprocessor may be further configured to determine the respectivedetermined coefficient based on the switching delay and the frame rate.

The first image frame captured by the first camera and the second imageframe captured by the second camera may be of a same scene, and theprocessor may be further configured to generate a photometric alignmentcoefficient based on the at least one transition parameter and generatethe at least one intermediate image frame using the photometricalignment coefficient.

The processor may be further configured to determine a transitioncoefficient for each intermediate image frame from among the at leastone intermediate image frame and generate each intermediate image framebased on the photometric alignment coefficient and on the respectivedetermined transition coefficient.

The memory may be further configured to store a switching delay betweenthe first camera and the second camera and a frame rate, and theprocessor may be further configured to determine the respectivedetermined transition coefficient based on the switching delay and theframe rate.

The first image frame captured by the first camera and the second imageframe captured by the second camera may be of a same scene, and theprocessor may be further configured to generate a color alignmentcoefficient based on the at least one transition parameter and generatethe at least one intermediate image frame using the color alignmentcoefficient.

The processor may be further configured to determine a transitioncoefficient for each intermediate image frame from among the at leastone intermediate image frame and generate each intermediate image framebased on the color alignment coefficient and on the respectivedetermined transition coefficient.

The memory may be further configured to store a switching delay betweenthe first camera and the second camera and a frame rate, and theprocessor may be further configured to determine the respectivedetermined transition coefficient based on the switching delay and theframe rate.

BRIEF DESCRIPTION OF FIGURES

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is an example illustration in which an electronic device switchesbetween a first lens and a second lens, according to an embodiment;

FIG. 2 shows various hardware components of the electronic device,according to an embodiment;

FIG. 3 shows various hardware components of a processor included in theelectronic device, according to an embodiment;

FIG. 4 is an example scenario in which a frame generation enginecomputes a number of frames to be generated, according to an embodiment;

FIG. 5 is a flow chart illustrating a method for switching between thefirst lens and the second lens in the electronic device, according to anembodiment;

FIG. 6 and FIG. 7 are example scenarios in which seamless transitionbetween the lenses are depicted, according to an embodiment;

FIG. 8 is an example flow chart illustrating various operations forswitching between the first lens and the second lens in the electronicdevice, according to an embodiment;

FIG. 9 is an example scenario in which the electronic device computesthe spatial alignment data, according to an embodiment;

FIG. 10 is an example scenario in which the electronic device computesthe spatial alignment data with respect to position (i.e., frame centeralignment) and scale (i.e., size of objects), according to anembodiment;

FIG. 11 is an example scenario in which the electronic device computesthe photometric alignment data, according to an embodiment;

FIG. 12 is an example scenario in which the electronic device determinesthe color alignment data, according to an embodiment;

FIG. 13 is an example flow chart illustrating various operations forgenerating the intermediate frames for smooth transformation from thefirst lens to the second lens, according to an embodiment;

FIG. 14 is an example flow chart illustrating various processes forperforming the spatial alignment procedure, according to an embodiment;

FIG. 15 is an example flow chart illustrating various processes forperforming the photometric alignment procedure, according to anembodiment;

FIG. 16 is an example flow chart illustrating various processes forperforming the color alignment procedure, according to an embodiment;and

FIG. 17 is an example flow chart illustrating various processes fordetermining the number of frame generation, according to an embodiment.

DETAILED DESCRIPTION

The embodiments herein and the various features and advantageous detailsthereof are explained more fully with reference to the non-limitingembodiments that are illustrated in the accompanying drawings anddetailed in the following description. Descriptions of well-knowncomponents and processing techniques are omitted so as to notunnecessarily obscure the embodiments herein. Also, the variousembodiments described herein are not necessarily mutually exclusive, assome embodiments can be combined with one or more other embodiments toform new embodiments. The term “or” as used herein, refers to anon-exclusive or, unless otherwise indicated. The examples used hereinare intended merely to facilitate an understanding of ways in which theembodiments herein can be practiced and to further enable those skilledin the art to practice the embodiments herein. Accordingly, the examplesshould not be construed as limiting the scope of the embodiments herein.

Embodiments may be described and illustrated in terms of blocks whichcarry out a described function or functions. These blocks, which may bereferred to herein as units or modules or the like, are physicallyimplemented by analog or digital circuits such as logic gates,integrated circuits, microprocessors, microcontrollers, memory circuits,passive electronic components, active electronic components, opticalcomponents, hardwired circuits, or the like, and may optionally bedriven by firmware and software. The circuits may, for example, beembodied in one or more semiconductor chips, or on substrate supportssuch as printed circuit boards and the like. The circuits constituting ablock may be implemented by dedicated hardware, or by a processor (e.g.,one or more programmed microprocessors and associated circuitry), or bya combination of dedicated hardware to perform some functions of theblock and a processor to perform other functions of the block. Eachblock of the embodiments may be physically separated into two or moreinteracting and discrete blocks without departing from the scope of thedisclosure. Likewise, the blocks of the embodiments may be physicallycombined into more complex blocks without departing from the scope ofthe disclosure.

The accompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the present disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings. Although the terms first, second,etc. may be used herein to describe various elements, these elementsshould not be limited by these terms. These terms are generally onlyused to distinguish one element from another.

Accordingly embodiments herein achieve a method for switching between afirst lens and a second lens in an electronic device. The methodincludes displaying, by the electronic device, a first frame showing afield of view (FOV) of the first lens. Further, the method includesdetecting, by the electronic device, an event to switch from the firstlens to the second lens. Further, the method includes generating, by theelectronic device, at least one intermediate frame for smoothtransformation from the first frame to a second frame showing a FOV ofthe second lens. Further, the method includes switching, by theelectronic device, from the first lens to the second lens and displayingthe second frame showing the FOV of the second lens. The at least oneintermediate frame is displayed between the first frame and the secondframe while the switching is performed.

Unlike conventional methods and systems, the electronic device generatesintermediate frames based on offline information computed relating tospatial transformation, photometric and color alignment. The generatedframes smoothly transform between the first lens preview (i.e., sourcelens preview) to a second lens preview (i.e., destination lens preview).The electronic device utilizes the source frame, offline spatialalignment data, photometric data and color alignment data to performthis transformation. This results in enhancing the user experience.

The electronic device switches from displaying a frame from one camerato a frame from another camera. The transition table containspre-calculated and pre-calibrated information for various transitions(e.g., wide to ultra-wide, tele to wide, wide to tele, etc.). Theinformation includes precise switching delay, FOV of lenses, color andsaturation profiles of the cameras, etc. The electronic device utilizesthe frames from the single camera and the transition table to generate afinal output of intermediate frames. For example, the electronic devicecomputes an interval between successive frames, the scale and positiontransformation for each generated frame, the color correction for eachframe, and the photometric correction for each frame. This results inenhancing the user experience.

FIG. 1 is an example illustration showing a process by which anelectronic device (100) switches between a first lens and a second lens,according to an embodiment. The electronic device (100) can be, forexample, but not limited to a cellular phone, a smart phone, a PersonalDigital Assistant (PDA), a tablet computer, a laptop computer, a smartwatch, an Internet of Things (IoT) device, a multi-camera system or thelike. The first lens and the second lens can be, for example, but notlimited to a wide lens, an ultra-wide lens, a tele-lens or the like.

In an embodiment, the electronic device (100) is configured to display afirst frame showing a FOV of the first lens. Further, the electronicdevice (100) is configured to detect an event to switch from the firstlens to the second lens. Further, the electronic device (100) isconfigured to generate at least one intermediate frame for smoothtransformation from the first lens to the second lens.

In an embodiment, the at least one intermediate frame is generated forsmooth transformation from the first lens to the second lens bydetermining a lens switching delay from a first frame showing the FOV ofthe first lens to a second frame showing a FOV of the second lens,detecting at least one lens transition parameter to generate the atleast one intermediate frame, obtaining at least one of a spatialalignment data, a photometric alignment data and a color alignment data,and generating the at least one intermediate frame between the firstframe and the second frame based on the at least one lens transitionparameter, the lens switching delay, and at least one of the spatialalignment data, the photometric alignment data and the color alignmentdata.

In an embodiment, the spatial alignment data is obtained by capturing asingle frame associated with the first lens and a single frameassociated with the second lens when the electronic device (100) is inan idle mode, resizing the single frame associated with the first lensand the single frame associated with the second lens into a previewresolution size, computing feature points in the single frame associatedwith the first lens and the single frame associated with the secondlens, computing a transformation matrix using a Homography relationshipbetween the single frame associated with the first lens and the singleframe associated with the second lens, wherein the transformation matrixincludes a scaling of the single frame associated with the first lensand the single frame associated with the second lens, a rotation of thesingle frame associated with the first lens and the single frameassociated with the second lens and a translation of the of the singleframe associated with the first lens and the single frame associatedwith the second lens, and obtaining the spatial alignment data using thetransformation matrix. The detailed operations of the spatial alignmentprocedure are explained in FIG. 14 .

In an embodiment, the photometric alignment data is obtained bycomputing a transformation matrix for the generated frame, computing acorrection factor based on the transformation matrix, and obtaining thephotometric alignment data based on the correction factor. The detailedoperations of the photometric alignment procedure are explained in FIG.15 .

In an embodiment, the color alignment data is obtained by computing atransformation matrix for the generated frame, computing a correctionfactor for the color alignment data based on the transformation matrix,and obtaining the color alignment data based on the correction factor.The detailed operations of the color alignment procedure are explainedin FIG. 16 .

In an embodiment, the at least one lens transition parameter is anF-value of the first lens, a FOV of the first lens, a color profile ofthe first lens, a saturation profile of the first lens, an F-value ofthe second lens, a FOV of the second lens, a color profile of the secondlens, a saturation profile of the second lens, a scale factor betweenthe first lens and the second lens, a single scale factor of acombination of the first lens and the second lens, a pivot between thefirst lens and the second lens, and a single pivot value of acombination of the first lens and the second lens.

In an embodiment, the at least one intermediate frame is at least one ofspatially aligned with respect to the first frame and the second frame,photometrically aligned with respect to the first frame and the secondframe and color aligned with respect to the first frame and the secondframe. The detailed operations of the intermediate frames generated forsmooth transformation from the first lens (150) to the second lens (160)are explained in FIG. 13 .

Further, the electronic device (100) is configured to switch from thefirst lens to the second lens and display the second frame showing theFOV of the second lens. The at least one intermediate frame is displayedbetween the first frame and the second frame while the switching isperformed. In other words, the at least one intermediate frame isdisplayed after the first frame is displayed and before the second frameis displayed.

In an embodiment, the at least one intermediate frame to be generated isdetermined by determining the spatial alignment data using the lenstransition parameter, and determining the at least one intermediateframe to be generated based on the determined spatial alignment data andthe lens transition parameter.

In an embodiment, the seamless transitions between the lenses areillustrated in FIG. 6 and FIG. 7 .

For example, as shown in FIG. 7 , the user of the electronic device(100) invokes the pinch-in-zoom on the image with the wider FOV to causea smooth transition to an image with a narrower FOV when enough detailsare not available in the wide FOV image. Similarly, the user of theelectronic device (100) invokes the pinch-out-zoom on the image with thenarrower FOV to cause a smooth transition to an image with wider FOVwhen enough details are not available in the narrow FOV image.

FIG. 2 shows various hardware components of the electronic device (100),according to an embodiment as disclosed herein. In an embodiment, theelectronic device (100) includes a processor (110), a communicator(120), a memory (130), a display (140), the first lens (150), the secondlens (160), and an application (170). The processor (110) is providedwith the communicator (120), the memory (130), the display (140), thefirst lens (150), the second lens (160), and the application (170). Theapplication (170) can be, for example, but not limited to a beautyrelated application, camera application, health related application orthe like.

In an embodiment, the processor (110) is configured to display the firstframe showing the FOV of the first lens (150). Further, the processor(110) is configured to detect an event that causes a switch from thefirst lens (150) to the second lens (160). Further, the processor (150)is configured to generate at least one intermediate frame for smoothtransformation from the first lens (150) to the second lens (160).Further, the processor (110) is configured to switch from the first lens(150) to the second lens (160) and display the second frame showing theFOV of the second lens (160). The at least one intermediate frame isdisplayed, on the display (140), between the first frame and the secondframe while the switching is performed. The at least one intermediateframe displayed on the display (140) may be visible to the user or itmay not be visible to the user.

The processor (110) is configured to execute instructions stored in thememory (130) and to perform various processes. The communicator (120) isconfigured for communicating internally between internal hardwarecomponents and with external devices via one or more networks.

The memory (130) also stores instructions to be executed by theprocessor (110). The memory (130) may include non-volatile storageelements. Examples of such non-volatile storage elements may includemagnetic hard discs, optical discs, floppy discs, flash memories, orforms of electrically programmable memories (EPROM) or electricallyerasable and programmable (EEPROM) memories. In addition, the memory(130) may, in some examples, be considered a non-transitory storagemedium. The term “non-transitory” may indicate that the storage mediumis not embodied in a carrier wave or a propagated signal. However, theterm “non-transitory” should not be interpreted that the memory (130) isnon-movable. In some examples, the memory (130) can be configured tostore larger amounts of information than the memory. In certainexamples, a non-transitory storage medium may store data that can, overtime, change (e.g., in Random Access Memory (RAM) or cache).

Although FIG. 2 shows various hardware components of the electronicdevice (100) but it is to be understood that other embodiments are notlimited thereon. In other embodiments, the electronic device (100) mayinclude fewer or more components. Further, the labels or names of thecomponents are used only for illustrative purpose and do not limit thescope of the disclosure. One or more components can be combined togetherto perform the same or a substantially similar function to switchbetween the first lens (150) and the second lens (160) in the electronicdevice (100).

FIG. 3 shows various hardware components of the processor (110) includedin the electronic device (100), according to an embodiment. In anembodiment, the processor (110) includes an event detector (110 a), aframe generation engine (110 b), a lens switching delay determinationengine (110 c), a spatial alignment data determination engine (110 d), aphotometric alignment data determination engine (110 e) and a coloralignment data determination engine (1100.

In an embodiment, the event detector (110 a) is configured to displaythe first frame showing the FOV of the first lens (150) and detect theevent that causes the switch from the first lens (150) to the secondlens (160). Further, the frame generation engine (110 b) is configuredto generate at least one intermediate frame for smooth transformationfrom the first lens (150) (i.e., from the first frame showing the FOV ofthe first lens) to the second lens (160) (i.e., to a second frameshowing the FOV of the second lens) using the lens switching delaydetermination engine (110 c), the spatial alignment data determinationengine (110 d), the photometric alignment data determination engine (110e) and the color alignment data determination engine (110 f). Further,the frame generation engine (110 b) is configured to switch from thefirst lens (150) to the second lens (160) and display the second frameshowing the FOV of the second lens (160). The at least one intermediateframe is displayed between the first frame and the second frame whilethe switching is performed.

The spatial alignment data determination engine (110 d) is configured tohandle the spatial alignment mismatch between the first and secondframes. The photometric alignment data determination engine (110 e) isconfigured to handle the different exposure between the first and secondframes. The color alignment data determination engine (1100 isconfigured to handle the color difference between the first and secondframes.

Although FIG. 3 shows various hardware components of the processor(110), it is to be understood that other embodiments are not limitedthereon. In other embodiments, the processor (110) may include fewer ormore components. Further, the labels or names of the components are usedonly for illustrative purpose and do not limit the scope of thedisclosure. One or more components can be combined together to performthe same or a substantially similar function to switch between the firstlens (150) and the second lens (160) in the electronic device (100).

FIG. 4 is an example scenario in which the frame generation engine (110b) computes a number frames to be generated, according to an embodiment.The frame generation engine (110 b) utilizes a transition table whichholds pre-calculated and pre-calibrated information for varioustransitions (e.g., wide lens to ultra-wide lens, tele-lens to wide lens,wide lens to tele-lens, etc.). The information includes switching delay,scale factor, color and saturation profiles of the lenses (150 and 160),etc. Using the transition table, the frame generation engine (110 b)computes the number frames to be generated for display.

In an example, the transition tables 1-6 are example tables for theelectronic device with three lenses (e.g., Ultra-wide lens, wide lens,tele-lens or the like). For each combination of lens transition thetransition table shows the corresponding transition parameters used inthe transition. A brief explanation follows of the parameters shown inthe transition table.

The “Enabled” parameter indicates whether the transition table isenabled or disabled (depending a lens configuration of the electronicdevice (100)). The “Switching delay” parameter indicates a delay betweenthe frame of the source lens and the frame of the target lens. The“Scale Factor X” parameter indicates an X Scale factor between thesource lens and the target lens. The “Scale Factor Y” parameterindicates a Y Scale factor between source and target lens. The “Pivot X”parameter indicates an X value of the pivot point for transition betweensource and target lens frames. The “Pivot Y” parameter indicates a Yvalue of the pivot point for transition between source and target lensframes. The “Brightness” parameter indicates a Brightness profile of thetarget lens frame, expressed in terms of mean and standard deviation.The “Color” parameter indicates a Color profile of the target lensframe, expressed in terms of mean and standard deviation. The pivotpoint is the point between source and target lens frames which remainsconstant during transition and may be specified using X and Ycoordinates.

Transition Table 1 for Ultra-wide lens to Wide lens Ultra-wide lens toWide lens Enabled True Switching delay 700 Scale Factor X 1.6 ScaleFactor Y 1.6 Pivot X 0.55 Pivot Y 0.5 Brightness 1.2, 0.2 Color 1.1, 0.1

Transition Table 2 for Ultra-wide lens to Tele-lens Ultra-wide lens toTele-lens Enabled True Switching delay 700 Scale Factor X 2.77 ScaleFactor Y 2.77 Pivot X 0.55 Pivot Y 0.5 Brightness 1.25, 0.2 Color 1.15,0.2

Transition Table 3 for Wide lens to Tele-lens Wide lens to Tele-lensEnabled True Switching delay 800 Scale Factor X 1.73 Scale Factor Y 1.73Pivot X 0.55 Pivot Y 0.5 Brightness 1.25, 0.2 Color 1.15, 0.2

Transition Table 4 for Wide lens to Ultra-wide lens Wide lens toUltra-wide lens Enabled True Switching delay 750 Scale Factor X 0.625Scale Factor Y 0.625 Pivot X 0.55 Pivot Y 0.5 Brightness 1.2, .02 Color1.1, 0.2

Transition Table 5 for Tele-lens to Ultra-wide lens Tele-lens toUltra-wide lens Enabled True Switching delay 650 Scale Factor X 0.361Scale Factor Y 0.361 Pivot X 0.55 Pivot Y 0.5 Brightness 1.25, 0.2 Color1.15, 0.2

Transition Table 6 for Tele-lens to Wide lens Tele-lens to Wide lensEnabled True Switching delay 700 Scale Factor X 0.578 Scale Factor Y0.578 Pivot X 0.55 Pivot Y 0.5 Brightness 1.25, 0.2 Color 1.15, 0.2

The transition tables 1-6 are only examples and are provided for thepurpose of understanding the transition parameters. Further, values forthe transition tables 1-6 may be varied based on at least one of theuser setting, an original equipment manufacturer (OEM), and aconfiguration of the electronic device (100).

FIG. 5 is a flow chart (500) illustrating a method for switching betweenthe first lens (150) and the second lens (160) in the electronic device(100), according to an embodiment. The operations (502-508) areperformed by the processor (110).

At 502, the method includes displaying the first frame showing the FOVof the first lens (150). At 504, the method includes detecting the eventthat causes a switch from the first lens (150) to the second lens (160).At 506, the method includes generating at least one intermediate framefor smooth transformation from the first lens (150) to the second lens(160). At 508, the method includes switching from the first lens (150)to the second lens (160) and displaying the second frame showing the FOVof the second lens (160). The at least one intermediate frame isdisplayed between the first frame and the second frame while theswitching is performed.

FIG. 8 is an example flow chart illustrating various operations forswitching between the first lens (150) and the second lens (160) in theelectronic device (100), according to an embodiment. At 802, theelectronic device (100) obtains the frame (i.e., the last frame from thefirst lens FOV and the first frame from the second lens FOV). At 804,the electronic device (100) generates the intermediate frames with thespatial alignment data based on the last frame of the first lens FOV andfirst frame of the second lens FOV. At 806, the electronic device (100)generates intermediate frames with the photometric alignment data. At808, the electronic device (100) generates intermediate frames with thecolor alignment data. At 810, the electronic device (100) combines theframes. At 812, the electronic device (100) renders the frames.

FIG. 9 is an example scenario in which the electronic device (100)computes the spatial alignment data, according to an embodiment.

For example, the electronic device (100) captures the pair of wide andultra wide frames keeping the electronic device stationary. Further, theelectronic device (100) resizes both images to preview resolution.Further, the electronic device (100) computes corner points in bothimages. Further, the electronic device (100) computes the transformationmatrix using Homography. Here, the Homography is a transformation matrix(e.g., a 3×3 matrix) that maps the points in one image to thecorresponding points in the other image. When applied to the sourceframe data, the transformation matrix effects scaling, rotation andtranslation of the source frame data. For spatial alignment, theelectronic device (100) needs scale and pivot data. Further, theelectronic device (100) constructs the transformation matrix using onlyscale and pivot data. In an example, below matrix is used for computingthe spatial alignment data.

$\begin{bmatrix}1 & 0 & p_{x} \\0 & 1 & p_{y} \\0 & 0 & 1\end{bmatrix} \cdot \begin{bmatrix}s_{x} & 0 & 0 \\0 & s_{y} & 0 \\0 & 0 & 1\end{bmatrix} \cdot {{\begin{bmatrix}1 & 0 & p_{x} \\0 & 1 & p_{y} \\0 & 0 & 1\end{bmatrix} = \left. \begin{bmatrix}s_{x} & 0 & {p_{x}\left( {1 - s_{x}} \right)} \\0 & s_{y} & {p_{y}\left( {1 - s_{y}} \right)} \\0 & 0 & 1\end{bmatrix}\leftrightarrow{\begin{bmatrix}s_{x} & 0 & t_{x} \\0 & s_{y} & t_{y} \\0 & 0 & 1\end{bmatrix}} \right.}}$

In this example, s_(x) and s_(y) represent scale factors X and Y andp_(x) and p_(y) represent pivot factors X and Y as described withreference to the Transition Tables above.

FIG. 10 is an example scenario in which the electronic device (100)computes the spatial alignment data with respect to position (i.e.,frame center alignment) and scale (i.e., size of objects), according toan embodiment.

The center point of the frame is different in the Ultra-wide frame andthe wide frame (for example, the crosshair position on the displayedbottle is different in the Ultra-wide frame and the wide frame). In anembodiment, the spatial alignment data gradually shifts the center fromthe Ultra-wide center to the wide center using the generatedintermediate frames. The scale of the Ultra-wide frame and the wideframe are different (for example, the size of the bottle is smaller inthe Ultra-wide frame than in the wide frame). Hence, the scale as wellis shifted gradually from the Ultra-wide scale to the wide scale usingthe generated intermediate frames.

FIG. 11 is an example scenario in which the electronic device (100)computes the photometric alignment data, according to an embodiment.

In the left side, the photometric histogram of the last frame from thefirst lens (i.e., a wide lens) is shown. The last frame from the firstlens is used as a reference image for photometric alignment. In theright side, the first preview frame from the second lens (i.e., theultra-wide lens) is shown. As shown in FIG. 11 , the photometrichistogram of the first preview frame from the second lens is differentfrom the photometric histogram of the last frame from the first lens.The electronic device (100) gradually aligns the intermediate framesphotometrically to the target frame using the photometric alignment datadetermination engine (110 e). The frames from the first lens and thesecond lens are photometrically different as seen from the histogramshift. Without photometric alignment, there will be a sudden change inbrightness.

FIG. 12 is an example scenario in which the electronic device (100)determines the color alignment data, according to an embodiment.

Consider, the reference image for the color alignment data is shown inthe left side of the FIG. 12 (i.e., last frame from the first lens(i.e., wide lens). The electronic device (100) generates the frameswithout photometric alignment with lower saturation. After transition,the electronic device (100) shows the first preview frame from thesecond lens (i.e., Ultra-Wide lens). The electronic device (100)generates the frames with photometric alignment data along with gradualsaturation change. The electronic device (100) modifies the color of theintermediate frames such that the color changes gradually for seamlesstransition using the color alignment data determination engine (1100.

FIG. 13 is an example flow chart (1300) illustrating various operationsfor generating the intermediate frames for smooth transformation fromthe first lens (150) to the second lens (160), according to anembodiment. The operations (1302-1318) are performed by the framegeneration engine (110 b).

At 1302, the user of the electronic device (100) initiates a smoothtransformation from the first lens (150) to the second lens (160). At1304, the electronic device (100) computes the alignment parametersusing the lens transition parameter from a corresponding transitiontable. At 1306, the electronic device (100) generates frame indices foreach of the intermediate frames to be generated. The number ofintermediate frames N to be generated is determined based on a switchingdelay (TSD) and a target frame rate (FPS) as described below withreference to FIG. 14 . At 1308, the electronic device (100) compares aframe number to a frame number N of a last frame. If the frame number isnot less than that of the last frame, at 1318, the method will stop. Ifthe frame number is less than that of the last frame, then at 1310, theelectronic device (100) performs the spatial alignment procedure. At1312, the electronic device (100) performs the photometric alignmentprocedure. At 1314, the electronic device (100) performs the performcolor alignment procedure. At 1316, the electronic device (100) displaysthe frames by combining the frames after performing the spatialalignment procedure, the photometric alignment procedure, and the coloralignment procedure.

FIG. 14 is an example flow chart (1400) illustrating various processesfor performing the spatial alignment procedure, according to anembodiment. The operations (1402-1416) are performed by the spatialalignment data determination engine (110 d).

At 1402, the electronic device (100) obtains the transition tableinformation. At 1404, the electronic device (100) computes the number offrames to be generated (i.e., N=TSD/FPS). At 1406, the electronic device(100) determines whether a frame number is greater than that of a lastframe. If the frame number is greater than that of the last frame, at1416, the method will stop. If the frame number is not greater than lastframe, then at 1408, the electronic device (100) computes αf (i.e.,αf=F(f, N, Mode)). Here, the term αf is a coefficient to be used whendetermining the transformation matrix for frame f. For frame f and totalnumber of frames N, αf=(N−f)/N. At 1410, the electronic device (100)computes the transformation matrix (i.e., Mf=αf*Z+(1−af)*T). Here, thematrix Z is the transformation matrix for frame 0 and the matrix T isthe transformation matrix for frame N as described below. At 1412, theelectronic device (100) performs an affine transformation with thedetermined transformation matrix. At 1414, the electronic device (100)displays the frames.

In an example, working of the frame generation engine (110 b) withrespect to the spatial alignment data is illustrated below:

Spatial Alignment Example:

The transformation matrix may be computed using the at least one lenstransition parameter identified from the transition table information.

Consider, the switching delay (ms): T_(SD), Target Frame Rate (FPS): F,then Frames to be generated (N)=T_(SD)/F

Transformation matrix for Frame 0

$(Z) = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}$

Transformation matrix for Frame

${N(T)} = \begin{bmatrix}s_{x} & 0 & {p_{x}\left( {1 - {sx}} \right)} \\0 & s_{y} & {p_{y}\left( {1 - {sy}} \right)} \\0 & 0 & 1\end{bmatrix}$

Then, the transformation matrix for each generated frame is,M _(f) =αf*Z+(1−αf)*T

Where, f is frame number and α_(f)=F(f, N, Mode)

Consider an example wherein T_(SD)=720 ms, F=60 fps, then N=(720/60)=12,and

${Z = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}},{T = \begin{bmatrix}1.62 & 0 & {- 0.25} \\0 & 1.62 & {- 0.18} \\0 & 0 & 1\end{bmatrix}}$

α₁=0.92, α₂=0.83, α₃=0.75 . . . α_(N)=0

FIG. 15 is an example flow chart (1500) illustrating various processesfor performing the photometric alignment procedure, according to anembodiment. The operations (1502-1516) are performed by the photometricalignment data determination engine (110 e).

At 1502, the electronic device (100) obtains the transition tableinformation. At 1504, the electronic device (100) computes the number offrames to be generated (i.e., N=TSD/FPS). At 1506, the electronic device(100) determines whether a frame number is greater than that of a lastframe. If the frame number is greater than that of the last frame, at1516, the method will stop. If the frame number is not greater than thatof the last frame, then at 1508, the electronic device (100) computes αf(i.e., αf=F(f, N, Mode)). Here, the term αf is a coefficient to be usedwhen determining the transformation matrix for frame f. For frame f andtotal number of frames N, αf=(N−f)/N. At 1510, the electronic device(100) computes the intensity mean-SD Correction Pf=αf*Y+(1−af)*S. Here,Y is the correction factor for frame 0 and S is the correction factorfor frame N as described below. At 1512, the electronic device (100)provides the photometric alignment with the correction factor Pf. At1514, the electronic device (100) displays the frames.

In an example, working of the frame generation engine (110 b) withrespect to the photometric alignment data is illustrated below:

Consider, switching delay (ms): T_(SD), Target Frame Rate (FPS): F,Frames to be generated (N)=T_(SD)/F, Mean Correction factor: C_(Mean),Standard Deviation Correction factor: C_(STD)

Correction factor for Frame 0 (Y): Y_(mean)=1.0, Y_(STD)=1.0 andCorrection factor for Frame N (S): S_(mean)=C_(Mean), S_(STD)=C_(STD)

Then the correction factor P_(f) (i.e., [P_(fMean), P_(fSTD)]) for eachgenerated frame is,P _(f)=α_(f) *Y+(1−α_(f))*S

Where, f is the frame number and α_(f)=F(f, N, Mode)

Consider, T_(SD)=720 ms, F=60 fps, N=(720/60)=12, α₁=0.92, α₂=0.83,α₃=0.75 . . . α_(N)=0

Using the above formula, correction factors P₁, P₂, P₃ . . . . P_(N) foreach intermediate frame may be computed.

Photometric alignment is applied for generating each intermediate frameaccording to the following relation:

${L^{\prime}{f\left( {x,y} \right)}} = {\left\{ {{{Pf}{Mean}} + {\left( {\frac{L\left( {x,y} \right)}{{Lf}{Mean}} - 1} \right){Pf}{STD}}} \right\}{Lf}{Mean}}$

where,

L_(f) is the intensity channel of the frame f

LfMean is the mean intensity of the frame f

The electronic device (100) applies the same logic for the coloralignment data.

FIG. 16 is an example flow chart (1500) illustrating various processesfor performing the color alignment procedure, according to anembodiment. The operations (1602-1616) are performed by the coloralignment data determination engine (1100.

At 1602, the electronic device (100) obtains the transition tableinformation. At 1604, the electronic device (100) computes the number offrames to be generated (i.e., N=TSD/FPS). At 1606, the electronic device(100) determines whether a frame number is greater than that of a lastframe. If the frame number is greater than that of the last frame, at1616, the method will stop. If the frame number is not greater than thatof the last frame, then at 1608, the electronic device (100) computes αf(i.e., αf=F(f, N, Mode)). Here, the term αf is a coefficient to be usedwhen determining the transformation matrix for frame f. For frame f andtotal number of frames N, αf=(N−f)/N. At 1610, the electronic device(100) computes the color mean-SD correction Cf=αf*X+(1−af)*R. Here, X isthe correction factor for frame 0 and R is the correction factor forframe N as described below. At 1612, the electronic device (100)provides the color alignment data with the correction factor Cf. At1614, the electronic device (100) displays the frames.

FIG. 17 is an example flow chart (1700) illustrating various processesfor determining the number of frames to be generated, according to anembodiment. The operations (1702-1706) are performed by the framegeneration engine (110 b).

At 1702, the electronic device (100) obtains the transition tableinformation. At 1704, the electronic device (100) computes the number offrames to be generated (i.e., N=TSD/FPS). At 1706, the electronic device(100) displays the frames.

The various actions, acts, blocks, steps, or the like in the flow charts(500, 800, and 1300-1700) may be performed in the order presented, in adifferent order or simultaneously. Further, in some embodiments, some ofthe actions, acts, blocks, steps, or the like may be omitted, added,modified, skipped, or the like without departing from the scope of thedisclosure.

The embodiments disclosed herein can be implemented using at least onesoftware program running on at least one hardware device and performingnetwork management functions to control the elements.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the embodiments herein that others can, byapplying current knowledge, readily modify and/or adapt for variousapplications such specific embodiments without departing from thegeneric concept, and, therefore, such adaptations and modificationsshould and are intended to be comprehended within the meaning and rangeof equivalents of the disclosed embodiments. It is to be understood thatthe phraseology or terminology employed herein is for the purpose ofdescription and not of limitation. Therefore, while the embodimentsherein have been described in terms of preferred embodiments, thoseskilled in the art will recognize that the embodiments herein can bepracticed with modification within the spirit and scope of theembodiments as described herein.

We claim:
 1. A method for switching between a first lens and a secondlens in an electronic device, comprising: displaying, by the electronicdevice, a first frame captured by the first lens showing a field of view(FOV) of the first lens, wherein the electronic device includes thefirst lens with the first FOV, the second lens with a second FOV greaterthan the first FOV, and a third lens with a third FOV greater than thefirst FOV and less than the second FOV; detecting, by the electronicdevice, an event that causes the electronic device to transition fromdisplaying the first frame to displaying a second frame showing a FOV ofthe second lens; generating, by the electronic device and based on aswitching delay which is determined based on a time difference between afirst time at which the first frame is displayed and a second time atwhich the second frame is displayed, at least one intermediate frame fortransitioning from the first frame to the second frame; and switching,by the electronic device and based on the detecting the event, from thefirst lens to the second lens and displaying the second frame, whereinthe at least one intermediate frame is displayed after the displayingthe first frame and before the displaying of the second frame.
 2. Themethod of claim 1, wherein the generating the at least one intermediateframe comprises: identifying at least one transition parameter of thefirst lens and the second lens to generate the at least one intermediateframe; obtaining at least one from among a spatial alignment data, aphotometric alignment data and a color alignment data of the first lensand the second lens; and generating the at least one intermediate framefurther based on the at least one transition parameter, and at least onefrom among the spatial alignment data, the photometric alignment dataand the color alignment data.
 3. The method of claim 1, wherein the atleast one intermediate frame is at least one from among spatiallyaligned with respect to the first frame and the second frame,photometrically aligned with respect to the first frame and the secondframe and color aligned with respect to the first frame and the secondframe.
 4. The method of claim 2, further comprising determining the atleast one intermediate frame to be generated, wherein determining the atleast one intermediate frame to be generated comprises: determining thespatial alignment data using the at least one transition parameter; anddetermining the at least one intermediate frame to be generated based onthe determined spatial alignment data and the at least one transitionparameter.
 5. The method of claim 2, wherein the spatial alignment datais obtained by: capturing a first single frame associated with the firstlens and a second single frame of a same scene associated with thesecond lens when the electronic device is in an idle mode; resizing thefirst single frame and the second single frame into a preview resolutionsize; computing feature points in the first single frame and the secondsingle frame; computing a transformation matrix using the identified atleast one transition parameter and a Homography relationship between thefirst single frame and the second single frame, wherein thetransformation matrix includes a first scaling of the first singleframe, a second scaling of the second single frame, a first rotation ofthe first single frame, a second rotation of the second single frame, afirst translation of the first single frame, and a second translation ofthe second single frame; and obtaining the spatial alignment data usingthe transformation matrix.
 6. The method of claim 2, wherein thephotometric alignment data is obtained by: computing a transformationmatrix for the generated at least one intermediate frame; computing acorrection factor based on the transformation matrix; and obtaining thephotometric alignment data based on the correction factor.
 7. The methodof claim 2, wherein the color alignment data is obtained by: computing atransformation matrix for the generated at least one intermediate frame;computing a correction factor for the color alignment data based on thetransformation matrix; and obtaining the color alignment data based onthe correction factor.
 8. The method of claim 2, wherein the at leastone transition parameter includes an F-value of the first lens, the FOVof the first lens, a color profile of the first lens, a saturationprofile of the first lens, an F-value of the second lens, the FOV of thesecond lens, a color profile of the second lens, a saturation profile ofthe second lens, a scale factor of the first lens, a scale factor of thesecond lens, a scale factor between the first lens and the second lens,a single scale factor of a combination of the first lens and the secondlens, a pivot between the first lens and the second lens, and a singlepivot value of a combination of the first lens and the second lens. 9.An electronic device for switching between a first lens and a secondlens, comprising: a memory; a processor coupled with the memory, theprocessor being configured to: display a first frame captured by thefirst lens showing a field of view (FOV) of the first lens, wherein theelectronic device includes the first lens with the first FOV, the secondlens with a second FOV greater than the first FOV, and a third lens witha third FOV greater than the first FOV and less than the second FOV;detect an event that causes the electronic device to transition fromdisplaying the first frame to displaying a second frame showing a FOV ofthe second lens; generate, based on a switching delay which isdetermined based on a time difference between a first time at which thefirst frame is displayed and a second time at which the second frame isdisplayed, at least one intermediate frame for transitioning from thefirst frame to the second frame; and switch, based on detecting theevent, from the first lens to the second lens and display the secondframe, wherein the at least one intermediate frame is displayed afterthe first frame is displayed and before the second frame is displayed.10. The electronic device of claim 9, wherein the processor is furtherconfigured to: identify at least one transition parameter of the firstlens and the second lens to generate the at least one intermediateframe; obtain at least one from among a spatial alignment data, aphotometric alignment data and a color alignment data of the first lensand the second lens; and generate the at least one intermediate framefurther based on the at least one transition parameter, and at least onefrom among the spatial alignment data, the photometric alignment dataand the color alignment data.
 11. The electronic device of claim 9,wherein the at least one intermediate frame is at least one from amongspatially aligned with respect to the first frame and the second frame,photometrically aligned with respect to the first frame and the secondframe and color aligned with respect to the first frame and the secondframe.
 12. The electronic device of claim 10, wherein the processor isfurther configured to: determine the spatial alignment data using the atleast one transition parameter; and determine the at least oneintermediate frame to be generated based on the determined spatialalignment data and the at least one transition parameter.
 13. Theelectronic device of claim 10, wherein the processor is furtherconfigured to: capture a first single frame associated with the firstlens and a second single frame of a same scene associated with thesecond lens when the electronic device is in an idle mode; resize thefirst single frame and the second single frame into a preview resolutionsize; compute feature points in the first single frame and the secondsingle frame; compute a transformation matrix using the identified atleast one transition parameter and a Homography relationship between thefirst single frame and the second single frame, wherein thetransformation matrix includes a first scaling of the first singleframe, a second scaling of the second single frame, a first rotation ofthe first single frame, a second rotation of the second single frame, afirst translation of the of the first single frame, and a secondtranslation of the second single frame; and obtain the spatial alignmentdata using the transformation matrix.
 14. The electronic device of claim10, wherein the photometric alignment data is obtained by: computing atransformation matrix for the generated at least one intermediate frame;computing a correction factor based on the transformation matrix; andobtaining the photometric alignment data based on the correction factor.15. The electronic device of claim 10, wherein the color alignment datais obtained by: computing a transformation matrix for the generated atleast one intermediate frame; computing a correction factor for thecolor alignment data based on the transformation matrix; and obtainingthe color alignment data based on the correction factor.
 16. Theelectronic device of claim 10, wherein the at least one transitionparameter includes an F-value of the first lens, the FOV of the firstlens, a color profile of the first lens, a saturation profile of thefirst lens, an F-value of the second lens, the FOV of the second lens, acolor profile of the second lens, a saturation profile of the secondlens, a scale factor of the first lens, a scale factor of the secondlens, a scale factor between the first lens and the second lens, a scalefactor of a combination of the first lens and the second lens, a pivotbetween the first lens and the second lens, and a single pivot value ofa combination of the first lens and the second lens.